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DETAILED ACTION 

1. This action is responsive to application filed on Dec. 19 2001. Claims 1 - 66 are 
pending examination. 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

2. Claims 1 - 3, 5, 6 f 8, 10, 14 - 18 - 26, 28, 29, 31 - 37, 39, 42 - 46, 49. 50, 53 - 
56, 58 - 63 and 66 are rejected under 35 U.S.C. 102(e) as being anticipated by Britton 
et al U.S. Patent No.6,535,896 (referred to hereafter as Britton). 

Britton teaches the invention explicitly as claimed. Britton teaches systems , methods 
and computer program products for utilizing XML - based tools to tailor HTML - based 
web page content for display within various client devices (see abstract). 

As to claim 1, Britton teaches a method of transcoding information in a first 
markup language into a second markup language, the method comprising the steps of: 

responding to a request to view a Web page by retrieving information from said 
Web page, wherein said information is in a first markup language (see col. 3 lines 28 - 
39, Britton discloses requesting an HTML- based web page ) ; 
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normalizing said information(see col. 3 lines 28 - 43 Britton discloses 
transferring the HTML - based web page format to XML format ); 

determining a second markup language that can be used by a browser using 
device detection, wherein said browser is used by a computer that is to view said 
information ( see col. 3 lines 40 - 48,Britton discloses the modified portions of XML 
document are converted back to HTML format) ; and 

transcoding said information into said second markup language( see col. 3 lines 
40 - 48, Britton discloses the conversion of the document to HTML and sent the web 
page with modified content to a client device for display). 

As to claim 2, Britton teaches the method of claim 1 , further comprising the step 
of sending said information in said second markup language to said computer (see col. 
3 lines 40 - 48, Britton discloses the conversion of the document to HTML and sent the 
web page with modified content to a client device for display). 

As to claim 3, Britton teaches the method of claim 2, wherein said computer is a 
wireless mobile device (see col. 3 lines 28 - 38. Britton discloses sending of a web page 
to pervasive computing devices). 

As to claim 5, Britton teaches the method of claim 2 wherein the step of sending 
said information in said second markup language to said computer comprises sending 
said information to said computer using automatic page division (see col. 3 lines 40 - 
48, Britton discloses the conversion of the document to HTML and sent the web page 
with modified content to a client device for display). 
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As to claim 6, Britton teaches the method of claim 1 , wherein said step of 
transcoding comprises the steps of: 

selecting a Tenderer that is associated with said second markup language from a 
plurality of Tenderers associated with markup languages ( see col.3 lines 43 - 48. 
Britton discloses converting back to HTML format); 

sending said information through said Tenderer ( see col.. 3 lines 40 - 48, Britton 
discloses the conversion of the document to HTML and sent the web page with modified 
content to a client device for display); and 

transcoding said information into said second markup language using said 
rendered see col. 3 lines 40 - 48, Britton discloses the conversion of the document to 
HTML format). 

As to claim 8, Britton teaches the method of claim 1 , wherein said step of 
normalizing comprises the step of transcoding said information in said first markup 
language into an intermediate markup language (see col. 5 lines 1-11 Britton 
discloses that XSL can be used to transform an XML document into one HTML view). 

As to claim 10, Britton teaches the method of claim 1, wherein said second 
markup language cor/iprises the Extensible Markup Language (XML) (see col.6 lines 10 
- 14 Britton discloses HTML web page content format converted to XML format). 

As to claim 14,Britton teaches the method of claim 1, wherein said second 
markup language comprises the HyperText Markup Language (HTML) (see col. 3 lines 
40 49, Britton teaches HTML language). 
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As to claim 1 5, Britton teaches the method of claim 1 , wherein said steps of 
responding, normalizing, determining, and transcoding occur automatically ( see col. 3 
lines 40 - 48, Britton discloses the conversion of the document to HTML and sent the 
web page with modified content to a client device for display). 

As to claim 1 6. Britton teaches the method of claim 1 , wherein said first markup 
language comprises the HyperText Markup Language (HTML) (see col. 3 lines 40 49, 
Britton teaches HTML language). 

As to claim 17, Britton teaches the method of claim 1 , further comprising the step 
of sending said information in said second markup language to said computer over a 
system of networked computers (see fig. 2 Britton discloses a pervasive computing 
device in a network). 

As to claim 18. Britton teaches the method of claim 1 , wherein a first object 
embodies said information in said first markup language and said step of transcoding 
further comprises automatic object conversion of said first object to a second object 
embodying said information in said second markup language (see col. 3 lines 40-48, 
Britton discloses the conversion of the document to HTML format). 

As to claim 19, Britton teaches the method of claim 1 , further comprising 
providing an error logging system (see col. 4 lines 54 - 60 Britton discloses reporting 
errors). 

As to claim 20, Britton teaches the method of claim 1 , wherein said second 
markup language is a markup language other than the HyperText Markup Language 
(HTML) (see col. 5 lines 1-13 Britton discloses Extensible Stylesheet Language). 
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As to claim 21 , Britton teaches the method of claim 1 , wherein said device 
detection comprises referring to an HTTP user agent header field (see col. 6 line 66 - 
col. 35. Britton discloses the hypertext transfer protocol). 

As to claim 22, Britton teaches the method of claim 1 , wherein said device 
detection comprises detecting said browser and said computer using unique signature 
detection (See col. 3 lines 7-14, Britton discloses a modified web browser). 

As to claim 23, Britton teaches the method of claim 1 , further comprising dividing 
said information in said second language into at least two pages using automatic page 
division (see col. 3 lines 50 - 62 Britton discloses web pages having a mixture of 
formats to be converted to a single format). 

As to claim 24, Britton teaches a method of transcoding information in a first 
markup language into a second markup language, the method comprising the steps of: 

responding to a request to view a Web page via a computer (see col. 3 lines 28 - 
39, Britton discloses requesting an HTML- based web page ); 

retrieving information from said Web page, wherein said information is in a first 
markup language, normalizing said information (see col. 3 lines 28-43 Britton 
discloses transferring the HTML - based web page format to XML format ); and 

transcoding said information into a second markup language, wherein said 
computer is adapted for utilizing said second markup language( see col. 3 lines 40 - 48, 
Britton discloses the conversion of the document to HTML and sent the web page with 
modified content to a client device for display). 
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As to claim 25, Britton teaches the method of claim 24, wherein said step of 
normalizing comprises the step of transcoding said information in said first markup 
language into an intermediate markup language (see col. 5 lines 1-11 Britton 
discloses that XSL can be used to transform an XML document into one HTML view). 

As to claim 26, Britton teaches the method of claim 24, wherein said computer is 
a wireless mobile device (see col. 3 lines 28 - 38. Britton discloses sending of a web 
page to pervasive computing devices). 

As to claim 28, Britton teaches the method of claim 24, further comprising 
dividing said information in said second language into pages using automatic page 
division (see col. 3 lines 50 - 62 Britton discloses web pages having a mixture of 
formats to be converted to a single format). 

As to claim 29, Britton teaches the method of claim 24, wherein said step of 
transcoding comprises the steps of: 

determining said second markup language, wherein said computer is adapted 
for utilizing said second markup language( see col. 3 lines 40 - 48,Britton discloses the 
modified portions of XML document are converted back to HTML format); 

selecting a renderer that is associated with said second markup language from a 
plurality of Tenderers associated with markup languages( see col. 3 lines 40 - 48, 
Britton discloses the conversion of the document to HTML format); 

sending said information through said renderer ( see col. 3 lines 40 - 48, Britton 
discloses the conversion of the document to HTML and sent the web page with modified 
content to a client device for display) ; and 



Application/Control Number 10/029,700 Page 8 

Art Unit: 2157 

transcoding said information into said second markup language using said 
renderer ( see col. 3 lines 40 - 48, Britton discloses the conversion of the document to 
HTML format). 

As to claim 31 , Britton teaches the method of claim 24, wherein said steps of 
responding, retrieving, normalizing, and transcoding occur automatically (see col. 3 
lines 40 - 48, Britton discloses the conversion of the document to HTML and sent the 
web page with modified content to a client device for display). 

As to claim 32, Britton teaches the method of claim 24, wherein a first object 
embodies said information in said first markup language and said step of transcoding 
further comprises automatic object conversion of said first object to a second object 
embodying said information in said second markup language (see col. 3 lines 40-48, 
Britton discloses the conversion of the document to HTML format). 

As to claim 33, Britton teaches the method of claim 24, further comprising 
providing an error log that reports errors that occur during at least one of said steps of 
responding, retrieving, normalizing, and transcoding (see col. 3 lines 40 - 48, Britton 
discloses the conversion of the document to HTML and sent the web page with modified 
content to a client device for display). 

As to claim 34, Britton teaches the method of claim 24, wherein said second 
markup language is a markup language other than the Hypertext Markup Language 
(HTML) (see col. 5 lines 1-13 Britton discloses Extensible Stylesheet Language). 

As to claim 35, Britton teaches the method of claim 24, further comprising the 
steps of: detecting a browser of said computer; and determining said second markup 
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language that is used by said browser based on said step of detecting (see col. 6 line 
66 - col. 35, Britton discloses the hypertext transfer protocol). 

As to claim 36, Britton teaches a method of transcoding information in a first 
markup language into a second markup language, the method comprising the steps of: 

responding to a request to view a Web page language (see col. 3 lines 28 - 39, 
Britton discloses requesting an HTML- based web page ); 

retrieving information from said Web page, wherein said information is in a first 
markup language(see col. 3 lines 28 - 43 Britton discloses transferring the HTML - 
based web page format to XML format ); 

device detection to determine said second markup language that is used by said 
browser(See col. 3 lines 7 - 14, Britton discloses a modified web browser) ; and 

transcoding said information into a second markup language, wherein said 
computer is adapted for utilizing said second markup language( see col. 3 lines 40-48, 
Britton discloses the conversion of the document to HTML and sent the web page with 
modified content to a client device for display). 

As to claim 37, Britton teaches the method of claim 36, wherein said computer is 
a wireless mobile device (see col. 3 lines 28 - 38. Britton discloses sending of a web 
page to pervasive computing devices). 

As to claim 39, Britton teaches the method of claim 36, wherein said step of 
transcoding comprises the steps of: 
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selecting a renderer that is associated with said second markup language from a 
plurality of Tenderers associated with markup languages( see col. 3 lines 40 - 48, 
Britton discloses the conversion of the document to HTML format); 

sending said information through said renderer ( see col. 3 lines 40-48, Britton 
discloses the conversion of the document to HTML and sent the web page with modified 
content to a client device for display) ; and 

transcoding said information into said second markup language using said 
renderer ( see col. 3 lines 40 «- 48, Britton discloses the conversion of the document to 
HTML format). 

As to claim 41, Britton teaches the method of claim 36, wherein said steps of 
responding, retrieving, device detection and transcoding occur automatically (see col. 3 
lines 40 - 48, Britton discloses the conversion of the document to HTML and sent the 
web page with modified content to a client device for display). 

As to claim 42, Britton teaches the method of claim 36, further comprising 
dividing said information in said second language into pages using automatic page 
division (see col. 3 lines 40 - 48, Britton discloses the conversion of the document to 
HTML and sent the web page with modified content to a client device for display). 

As to claim 43, Britton teaches the method of claim 36, wherein a first object 
embodies said information in said first markup language and said step of transcoding 
further comprises automatic object conversion of said first object to a second object 
embodying said information in said second markup language (see col. 3 lines 40 - 48, 
Britton discloses the conversion of the document to HTML format). 
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As to claim 44, Britton teaches the method of claim 36, further comprising 
transcoding said information in said first markup language into an intermediate markup 
language prior to transcoding said information into second markup language (see col. 5 
lines 1-11 Britton discloses that XSL can be used to transform an XML document into 
one HTML view). 

As to claim 45, Britton teaches a system for viewing a Web page by a computer 
that utilizes a markup language, the system comprising: 

a computer, wherein said computer requests to view a Web page(see col. 3 lines 
28 - 39, Britton discloses requesting an HTML- based web page ); 

information from said Web page, wherein said information is in a first markup 
language; a device detector, wherein said device detector determines a second markup 
language that said computer utilizes( see col. 3 lines 40 - 48,Britton discloses the 
modified portions of XML document are converted back to HTML format); and 

a renderer, wherein said Tenderer transcodes said information into said second 
markup language, wherein said information is sent to said computer( see col. 3 lines 40 
- 48, Britton discloses the conversion of the document to HTML and sent the web page 
with modified content to a client device for display). 

As to claim 46, Britton teaches the system of claim 45, further comprising: 

a normalizer, wherein said normalizertranscodes said information in said first 
markup language into an intermediate markup language(see col. 3 lines 28 - 43 Britton 
discloses transferring the HTML - based web page format to XML format). 



Application/Control Number: 1 0/029,700 Page 1 2 

Art Unit: 2157 

As to claim 49, Britton teaches the system of claim 45, wherein said computer 
utilizes a markup language other than the HyperText Markup Language (HTML) (see 
col. 5 lines 1 - 13 Britton discloses Extensible Stylesheet Language). 

As to claim 50. Britton teaches the system of claim 45, wherein said computer is 
a wireless mobile device (see col. 3 lines 28 • 38. Britton discloses sending of a web 
page to pervasive computing devices). 

As to claim 53, Britton teaches the system of claim 45, wherein said information 
in said second markup language is sent to said computer over a system of networked 
computers (see fig. 2 Britton discloses a pervasive computing device in a network). 

As to claim 54, Britton teaches the system of claim 45, wherein a first object 
embodies said information in said first markup language and said Tenderer uses 
automatic object conversion to convert said first object to a second object embodying 
said information in said second markup language (see col. 3 lines 40 - 48, Britton 
discloses the conversion of the document to HTML format). 

As to claim 55, the system of claim 45, further comprising an error logging 
system (see col. 4 lines 54-60 Britton discloses reporting errors). 

As to claim 56, Britton teaches the system of claim 45, wherein said second 
markup language is a markup language other than the HyperText Markup Language 
(HTML) (see col. 5 lines 1-13 Britton discloses Extensible Stylesheet Language). 
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As to claim 57, Britton teaches the system of claim 45, wherein said device 
detector uses unique signature detection (See col. 3 lines 7 - 14, Britton discloses a 
modified web browser). 

As to claim 58, Britton teaches a system for viewing a Web page by a computer 
that utilizes a markup language other than the Hypertext Markup Language (HTML), 
the system comprising: 

a computer, wherein said computer requests to view a Web page (see col. 3 
lines 28 - 39, Britton discloses requesting an HTML- based web page ) ; 

information from said Web page, wherein said information is in a first markup 
language(see col. 3 lines 28 - 39, Britton discloses requesting an HTML- based web 
page ); 

a normalizes wherein said normalizer normalizes said information in said first 
markup language into an intermediate markup language(see col. 3 lines 28 - 43 Britton 
discloses transferring the HTML - based web page format to XML format ); and 

a Tenderer, wherein said renderer transcodes said information in said 
intermediate markup language into a second markup language, wherein said second 
markup language is a markup language that said computer utilizes and said second 
markup language is a markup language other than HTML (see col. 5 lines 1-11 Britton 
discloses that XSL can be used to transform an XML document into one HTML view). 

As to claim 59, Britton teaches the system of claim 58, further comprising: 
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a device detector, wherein said device detector determines said second markup 
language based on a browser of said computer (see col. 6 line 66 - col. 35, Britton 
discloses the hypertext transfer protocol). 

As to claim 60 t Britton teaches the system of claim 58, wherein said computer is 
a wireless mobile device (see col. 3 lines 28 - 38. Britton discloses sending of a web 
page to pervasive computing devices). 

As to claim 61, Britton teaches the system of claim 58, wherein a first object 
embodies said information in said first markup language and said tenderer uses 
automatic object conversion to convert said first object to a second object embodying 
. said information in said second markup language (see col. 3 lines 40 - 48, Britton 
discloses the conversion of the document to HTML format). 

As to claim 62, Britton teaches computer executable process steps operative to 
control a computer, stored on a computer readable medium, comprising: 

a plurality of steps to receive data required for subsequent calculations(see col. 3 
lines 28 - 43 Britton discloses transferring the HTML - based web page format to XML 
format ); and 

a plurality of steps to automatically transcode information in a first markup 
language into a second markup language, wherein said second markup language is 
automatically determined ( see col. 3 lines 40 - 48, Britton discloses the conversion of 
the document to HTML and sent the web page with modified content to a client device 
for display). 
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As to claim 63, Britton teaches the steps of claim 62, further comprising a step to 
automatically normalize said information in said first markup language prior to 
transcoding said information into said second markup language (see col. 3 lines 28-43 
Britton discloses transferring the HTML - based web page format to XML format). 

conversion of the document to HTML format). 

As to claim 66, Britton teaches the method of claim 64, further comprising 
dividing said information in said second language into pages using automatic page 
division (see col. 3 lines 50 - 62 Britton discloses web pages having a mixture of 
formats to be converted to a single format). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 4, 7, 27, 30, 38, 40, 47, 48, 51 , 52, 64 and 65 are rejected under 35 

U.S.C. 103(a) as being unpatentable over Britton in view of Vega - Garcia et al. 

U.S.Patent No. 6.839.734 (referred to hereafter as Vega-Garcia). 

As to claims 4, 7, 27, 30, 38, 40, 47, 48, 51 and 52, Britton teaches systems , 

methods and computer program products provided utilizing XML-based tools to tailor 
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HTML-based Web page content for display within various client devices. Moreover 
Britton teaches a method of transcoding information in a first markup language into a 
second markup language, the method comprising the steps of: 

responding to a request to view a Web page by retrieving information from said 
Web page, wherein said information is in a first markup language (see col. 3 lines 28 - 
39, Britton discloses requesting an HTML- based web page ) ; 

normalizing said information(see col. 3 lines 28-43 Britton discloses 
transferring the HTML - based web page format to XML format ); 

determining a second markup language that can be used by a browser using 
device detection, wherein said browser is used by a computer that is to view said 
information ( see col. 3 lines 40 - 48,Britton discloses the modified portions of XML 
document are converted back to HTML format) ; and 

transcoding said information into said second markup language( see col. 3 lines 
40 - 48, Britton discloses the conversion of the document to HTML and sent the web 
page with modified content to a client device for display). 

Britton does not teach streaming information in real time. However, Vega-Garcia 
teaches multimedia communications software with network streaming and multi-format 
conference (see col. 4 lines 20 - 27). It would have been obvious to one of the ordinary 
skill in the art to include streaming information in real time as in Britton's because doing 
so would enable a user to avoid delay entailed in downloading an entire file and then 
playing it . 
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As to claims 64 and 65, Britton teaches the invention as mentioned above. 
Britton does not teach streaming information in real time. However, Vega-Garcia 
teaches multimedia communications software with network streaming and multi-format 
conference (see col. 4 lines 20 - 27). It would have been obvious to one of the ordinary 
skill in the art to include streaming information in real time as in Britton's because doing 
so would enable a user to avoid delay entailed in downloading an entire file and then 
playing it . 

4. Claims 9 and 1 1-13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Britton. 

Britton teaches the intermediate markup language is XML and the second format is 
Html. Britton does not explicitly teach the intermediate markup language is XHTML, and 
the second markup language is WML, cHTML or HDML. Official notice is taken that it 
would have been obvious for one of the ordinary skill in the art at the time of the 
invention was made to modify Britton by using the markup languages because doing so 
allow the system to use different formats of markup languages that are extremely 
simple" dialect of language suitable for use on the World-Wide-Web and therefore does 
not require use of specialized software on the client device. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

- System and Method for Cooperative Client/Server Customization of Web 
pages by Fields et al. U.S. Patent No. 6,412,008. 
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- System In which a Proxy- Server Translates Information Received From the 
Internet Into A Form/Format readily Usable By Low Power Portable 
computers by Kiknis U.S. Patent No. 5,727,159. 

- System For Dynamically Transcoding Data Transmitted Between Computers 
by Tso et al U.S.Patent No. 6,421 ,733 

- An Active Transcoding Proxy to Support Mobile Web Access by Harini 
Bharadvaj ( University of Missouri- Columbia). 

Transcoding Internet Content For Heterogeneous Client Devices. By John R. 
Smith, Rakesh Mohan and Chung- Sheng Li, pages III-599 - III- 602.Any inquiry 
concerning this communication or earlier communications from the examiner should be 
directed to Sargon N Nano whose telephone number is (571 ) 272-4007. The examiner 
can normally be reached on 8 hour. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ario Etienne can be reached on (571) 272-4001. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
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Abstract 

Iti litis paper, we prtittit a pwsy bused system 
(MOWSBK) to support web browsing from mobile clients 
over wireless networks, Moirscr is a proxy agent between 
the mobile host and the web server, whicJi performs active 
transcoding of data on both upstream and downstream trdf- 
fjtc to present web information to the mobile- user according 
to the QoS'pdrameLers set by the nser. Activr. transcoding is 
def$ne<i us modifying the HT-TD stream in situ, and it is en- 
tirely transparent to the user. Further, our system does not 
pose any additional requirements on the mobile user. This 
is un improvement over other proxy based systems, which 
only transcodc images on Uic downstream and are mostly 
vol configurable. While developed for mobile users, such a 
system cun uciuulty be useful in uny low banilwidth sccnavio. 



1. Introduction 

With ad ever increasing amount of information "out 
there" on the. World Wide Web, and mobility becoming 
the need of the Hour, user* want to Have information at 
tBeir fingertips wherever they, may Be. Recent develop- 
ments in mobile computing and web technologies have re- 
sulted in an increase in trie number of *road warriors*, peo- 
ple who use mobile computers to access their business in- 
formation repositories, often through the .web. ^Eriese users 
need speedy access to thai Hata. One cannot expect them 
to wait for hours to download a web page containing a lot 
of multimedia data over a wireless network, most of which 
cannot: be handled by the small and "easy-to-carry" lap- 
top/palmtop computer anyway. Most mobile computers 
have very limited cpu, memory & disk resources. Mxfrey 
communicate oven wireless links which' are characterised by 
lower band widths, higher error rates, anil more frequent dis- 
connections. Moreover, in a mobile environment, changes 
to the network bandwidth and resources are very common. 
TltK.se are only Mime of the challenges of mobile computing, 
more of which arc discussed in f«t). In I he context of web 
browsing, web servers do uoi consider the network connec- 
tion between l hem and the client. They also have no con- 
cern for the hardware capabilities of the client. Tliry just 

* Hi is tv.»rk w;i»; *iip| Hirt-c't in part l».y an IBM Rieotty Dev-tJ- 
opiHuiit Awnnl i»» A. J«»shi 



return the document asked for assuming that the client is 
capable of properly receiving and displaying the data. Peo- 
ple are commonly creating web pages rich in multimedia 
data, pages with* several images and videos have become 
very common. Presenting all this multimedia-rich data over 
wireless networks to the mobile user, is a major challenge. 
The mobile computer either, does tint have the appropriate 
hardware or adequate bandwidth (or both.) to handle the 
content, and the web servers also cannot Be modified Co 
suit the mobile user. ^Therefore, the only, way of Bridging 
the gap between the Highly resource- ridi web senrey at one 
end and the highly resource-poor mobile client at the other, 
is by introducing a system in between wKicb modifies the 
contents on the wen and present it in the most appropriate 
form to the mobile user. Hence, the need foF middleware, 
which adapts to the mobile environment, gets the be3t of 
what is available on (lie web atid does noli pose any. addi- 
tional requirements on the mobile client. 

We Have devised such a middleware based solution which 
allows the user of a mobile computer to control the way 
in which the data from fcKr weh is retrieved, dynamically 
transform the data in a way that is transparent to the user, 
without requiring the mobile computer to do any additional 
work. We achieve this by introducing a proxy agent be- 
., I ween the mobile client and the web server. Our proxy, 
Mowser; lets the. user specify, ana* control- the viewing pref- 
erences and hardware capabilities of the client . transforms 
the data to and from the weh server, without requiring any 
change on the client. We talk' about related work in section 
2 and describe the software architecture of our system in 
section 3. In section 4. we describe the various methods of 
transforming data that we have used. The results of our 
experiments are presented in section 5 and we discuss it in 
section 6. 

2. Related 

TRc Glient-Rroxy.-Servcr model has begun to feature in 
many mobile applications to overcome the challenges faced 
in the mobile computing scenario. However, only some of 
them actually transform the stream between the client and 
the server. 

In the CJIoMop model described in (5). [I I J the proxy 
performs "distillation* of the document received from the 
server before sending it to the client. Distillation is defined 
line as highij lo>:>y, real-Lime. datatype->pecific compres- 
sion that preserves most of the semantic content of a docu- 
ment. The approach of transcoding image lilt s is similar to 
our approach lir*t outlined in [U], but video files arc handled 



differently. Whey, perform real-time transcoding of motion 
JPEG to sub-sampled H.261, while our approach, described 
later in this paper, represent* a video stream using repre- 
sentative frames. GloMop also allows refinement of selected 
portions of the document. 

The Mowgli model [V4] consists of two mediators, the 
Mowgli A : gent and the Mowgli Proxy located on the mo- 
bile host and the mobile-connection host respectively. They 
use the Mowgli HTTP protocol to communicate with each 
other, which reduces tli'c number of round- trips between 
client ancl server. A : specialized transport service, the 
Mowgli Data Channel Service is used for reliable communi- 
cation between the mobile- connection host and the mobile 
host. Mowgli WWW reduces the data transfer over the 
.wireless link in three ways: data compression, caching, and 
intelligent filtering, it only performs (3 IK to JKEG conver- 
sion, and large embedded images arc not transferred at all 
to the mobile node. 

Zenel proposes an architecture in [19] in which the proxy 
is made up of three components: High Level Proxy, bow 
Bevel Proxy and Event Manager. The High Level Proxy al- 
lows filters for application layer protocols llo lie downloaded 
dynamically from mobile host applications. 'Hie tow bevel 
Proxy is used to create and install (liters for the transport 
and network layers. A' control interface is provided for- the 
filters running within these proxies By the Even! Manager. 

HTTP transducers called OreOs (defined in [3]) are spe- 
cialized processing modules having tour classes of function; 
ality: filtering HTTP requests and responses, characterizing 
sets of messages, transforming message contents and addi- 
tional processing indicated by the messages, The "stream 
model" approach of signal processing is used to perform 
complicated processing. 

In the work of Sathyanarayanan «t. al, a module called 
Cellophane (17] on the client transforms HTTP requests 

" from Netscape into file operations on Odyssey [16] web ob- 
jects and makes use of Odyssey API to select fidelity levels 
for images which' are forwarded to a distillation server. The 
distillation server distills the images to the requested fidelity 

. before passing them back to the client. However, this ap- 
proach is specific to the Odyssey file system, and-re quires a 
modified version of the Met BSD kernel This also requires 
the addition of a modulo on the client. 

Intei s s Quidc Web Technology (7] which' sits on ISP 
servers compresses images by. selectively dropping bits or 
pixels out of an image using lossy compression techniques, 
thereby, speeding up the download ot graphically- rich web 
pages. It also caches data to overcome the problem of band- 
width bulge. This can be used only when the access to 
the Internet is through ISP, and the user requires a Java- 
enabled browser to have control over the compression. 

IBM's Web Express [6] consists of two components: AR- 
Tonr (Advanced Radio Communications on Tour J Gateway 
and AHTour Client. The Gateway provides secure, com- 
pressed data across the selected network with authentica- 
tion. It can automatically retrieve Web requests in the 
background while mobile users are performing other tasks. 

In our system., the proxy preforms active transcoding 
of HVJ.V requests from the client while sending it to the 
strrvttr. according to die preference* set by the mobile host. 
j?n that the document in ihe most rfiiitalue format U re- 
uieved. It ateo processes the received HTTP data before 
sending it to the mobile host if necessary. 



3. Software Architecture 



Mowser is a proxy H3£ER server agent {written in Peri) 
[18] which allows a mobile user to specify his or lier view- 
• ing preferences, based on the network connection and avail- 
able resources, and performs active transcoding of HTTP 
streams accordingly. The software architecture that we 
propose introduces a proxy server on each Mobile Support 
Station (MSS) to the basic MSS-MH (Mobile Host) model 
which accepts and stores the preferences for each' of its mo- 
bile Hosts, acts as the server to the mobile host, and as a 
client to the -WWW server. No modifications to the web 
client on the MH is required. So any :WWW browser that 
can handle forms and Has the provision of a proxy can be 
used. No additional software is required on the mobile host. 
Also, setting and updating preferences is done by just fill- 
ing up a OGI form on o Uftli at the web site maintained 
by. the proxy server. 

In the initial versions of Mowser [9],f £Qj, the proxy dealt 
with getting the viewing preferences for a MH from the user 
and storing it according Co its IB address. The current ver- 
sion also stores the accept headers that will be most suitable 
for tfie MH based on the preferences set by the user. The 
viewing preferences stored for eacb MH include a starting 
point, color capability, video resolution, sound capability, 
maximum allowed size for text, image, video, audio files 
and files of unknown type, and the. size reduction technique 
for image hies. Not all of these* variables are presently used 
by the proxy. The preferences can Be updated by the user of 
the MH whenever there is a change in Che network connec- 
tion or available resources. We are using an A pache HTTP 
server to store the preferences, ancl 601 scripts writfeu in 
Tel and Perl are used to update and save the preferences. 

Once the MR sets the Proxy server as its proxy, all com- 
munication between the MB and the WVKXV servers is di- 
rected through this proxy. When the proxy receives a re- 
quest from the MH, it looks up the preferences stored with 
the IP address of the MH and processes the request accord- 
ingly. Default preferences are used if: no preferences KaH 
been specified. The proxy processes requests to set pref- 
erences and the GET requests. AO other requests are for- 
warded to tlie target MfVVW server. In the next section, we 
r detail how (He proxy performs active transcoding of- HTTP 
' streams/ • ~r lZ !••*-•' ' ' * ; . , :* 



4. Active ^Transcoding 

Modifying the HTTP stream and changing its content in 
situ is called active transcoding. This is done dynamically 
without any user intervention. For example, if an image file 
does not meet the size or' color specifications, it is reduced 
before being sent to the MH (described io section 4.2). Sim- 
ilarly, sound files will not be sent to a MH with no sound 
capabilities, and so on. 

Traditionally, transcoding is a unidirectional process [9], 
(5],[6],(7]. In other words, the request from the client is 
passed as b to the target server, while the return stream's 
multimedia content is altered. In our work, we alter tlie 
request as well, so as to take advantage of some net-friendly 
features of I I'nrP/l.l. 

After setting preferences and making our MOWSER as 

proxy, the user can browse the web as s/he would with 
any web client. On receiving a request from the Mil, the 
pruxy fetches the preferences set l»y t.he MH and serves the 
MH with fihas in tlie most suitable format. Default pref- 
erences are used if no values had been set By tlie MH. We 
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process H'l?&> G&U requests received from the MH be- 
fore sending it to the WWW- server, and modify image and 
video files received from the WWW server before transmit- 
ting it Co the MH if: necessary. $b support MHs with very, 
limited resources and Hardware capabilities bice PDAs, we 
even parse (lie HTMU stream to remove cite acti ve couteul: 
and any tags that the MH cannot handle. The user may 
even choose to block any HTML; file greater than a given 
size. Witti transcoding Being done al two steps indepen- 
dently as shown in Figure J, we are making sure that we 
match the preferences of: the MH, while using the wired 
Band width in the most efficient manner. 

4.1. Transcoding of HT3?F? Requests 

HTTBjJl.l introduces the concept vf content negotia- 
tion. The basic idea is that a WWW server may have sev- 
eral different representations of a resource. For example, il 
may store a document as postscript or word of HTMLi, etc. 
&he server, ran automatically choose the file to send if the 
client send^ the preferred representations as part of each 
request. Most servers (e.g., apache), even Chough not fully 
I. T compliant , already snppoi£ content- negotiation ami will 
store fih.-s in several formats and in several variations of a 
format. We use this idea to get the file in the most ap- 
propriate format for the MH. For example, an image file 
may tie made availahle in varying resolutions hy the con- 
tent provider on the server. We request the server to send 
the image file which Has the resolution appropriate to the 
present QoS and dieiit parameters, by including Che prefer- 
ence in the request. This requires that the variants of a file 
have different mime types. For example, in our. experiments 
we liave used iinage^x-sgif to denote an image file with' very 
low resolution, image/x-mgif to denote one witK medium 
resolution and image/x-lgif to denote one with large resolu- 
tion. Wc Have also introduced video jfx-rmpg to denote rep- 
resentative frames of video files (discussed in section 1.3). 

ftny HTTP GST request received from the MH is 
munged to an HSETP i.i request and the complete URI 
is included in the request line. The Accept headers stored 
for the MH are then appended to the outgoing stream to 
request for the file in the format most suited for the MH. 
ft Host header is added to complete the HSBFP 1. 1 request. 
The server performs content negotiation and aenuV the file • 
which closely meets the format specified in trie request/ 
Thus, the process is transparent to the user, and works even 
if the request comes from an H32$Bgl.O compliant browser, 
like most present commercial systems. 

Pot example, for a MH host on a low Bandwidth line, 
the proxy may append the following Accept headers to the 
request after making it a HTTP t.l request: 

Accept: image/x-egif, vitfeo/x-rmpg 

For a MH like the Palm Pilot, which can handle only text 
and images, the proxy greatly, reduces the data transfer by 
selectively GETing the files. fPliat is, when Che proxy re 
ceives a GET request from a PDA*, it sends a HEAD request 
to the WWW server to get information ahout the content 
type of the file, and then GETs the file only, if the PD$ 
can handle it. For example, the proxy does not request for 
audio, video and application files for a PDA. Since a page 
has the URLs of additional files to be fetched embedded in 
it. wc cou Ul prevent the client on the MH from generating 
the additional GET requests and design the proxy to decide: 
whether to GET the file or not: by just looking at the exten- 
sion of (lie lil* nanirt in the embedded URI-s. However, thu 
content-type of the file id a better indicator of the format of 



the lite than the extension in the filename, though getting 
this information requires an additional HEAD request to 
he sent. 

4.2. Transcoding Image Sites 

When the proxy finds an image tag in the HTTP stream 
received from the server, it reads the Uftl of the image 
file to be fetched and first sends a HEAD request to the 
server. It checks the content-type and content-length in- 
formation received from the server. If tine content- length 
is small enough to be handled on the MH, tHe image file 
is sent to the MH unmodified. But if the image is larger 
than what can be Handled by Hie MH t it b reduced in size 
or color as requested by the MH. The image files are sealed 
down in size, or the number of colors is reduced, or both 
without sacrificing semantics. On an image map for in- 
stance, size is not changed, only, colors axe, to preserve the 
semantics. The content- type Information is used to decide 
the transformations that tHe image file has to go through. 
We convert all images to Be reduced to portable pixmap 
format for processing and then convert (hem back to gif 
format for displaying. Then the original ©RE in the image 
tag is replaced .with the ERL of the modified image stored 
locally by Che proxy and senr: to the MH. This makes fhe 
MH GET the modified image file from the proxy. To display 
images on PDAs, ttie proxy might have to reduce images to' 
2-bit gray scale and thumb size. 

4.3. Transcoding Video Data 

Unlike image data where transcoding steps are obvi- 
ous, video data represents a great challenge. Simple sub- 
sampling, as proposed in [5|. is still not adequate as some 
clients may not hare enough computational resources to do 
software decoding of MPEG or H.26J. We use the struc- 
ture inherent in video streams to do the transcoding. The 
structure of video is a hierarchy of the movie or episode. 
Tins hierarchy is segmeuts. scenes, and shots. Each seg- 
ment consists of sequence of scenes, each' scene consists of 
several shots, and each shot is composed of several frames 
which have similar' visual 'properties: ^Thos one* of these 
frames can be selected as a Representative -frame (Rfrarae) 
for the shot. 

We present tHe video to the user by the representative 
frames which are picked from each shot. Using techniques, 
we have developed [i] ; [8] to support content based access 
to networked video databases, Wc have used several fuzzy 
clustering algorithms, such as fuzzy c-mean, bard c-mean, 
fuzzy c- median, bard c- median and possioilisUc c-mean 
Mi[l3],[I2]. We use luminance and chrominance features, 
and l-norm and 2-norm distance measures (4], (8j in order 
to group the frames which have similar properties together. 
Each group is classified as one shot. We pick tlie frame 
that is closest to center of cadi group to be Rfeamc. Wc do 
not explicitly use any. scene change defection algorithms. 
The fuzzy techniques are used since frames can belong to 
the clusters to different degrees (membership values). Tra- 
ditional scene change algorithms, which insist on a frame 
belonging to only, one group, break down when confronted 
with gradual scene changes typically found in videos. While 
Kframes can be computed dynamically by the proxy, we feel 
that Trom a computational perspective, this should lie done 
at the server side. In fact., it. can be argued [lj,($j ihat 
these will typically be available at the server side already 
to support querying and browsing of the video database. 



4.4. 'Transcoding Active Content 

For mobile frusta wil.h' limited memory, and computa- 
tional resources, the user can decide not to receive any Java 
applets, JavaScripts, IWBScripts. etc. When such a prefer- 
ence is set, the document to Be transmitted is parsed and 
all the active content w eliminated before sending it to the 
MR. 3?hu is specifically, suitable for the PDAs which have 
very small disk space and low speed GPUa. Also, given re- 
strictions on tHe memory footprints of the applications that 
can reside on such machines, it is not clear, that Browsers 
will be able to support the virtual machines needed for ac- 
tive content languages such as Java. 

Often though, iiava (and JavaScript) are used to provide 
functionality, that can Be duplicated using CGI callbacks or 
server parsed HTME. ftn interesting option that wc wish 
to pursue Here is to see if we could have the forms capable 
client request for a CGI version instead of the JavaScript 
from the server. In other words, replace active content of 
a page with equivalent dynamic content. This feature will 
be supported via content negotiation, lake all content ne- 
gotiation, it assumes that the server provides alternative 
versions (CGI based vs Java Based) of a particular URB. 

.4.5. transcoding H3?MI 

We can reduce the computation on the Mil by parsing 
HTME tags on the proxv itself, rather that on the MH. We 
can eliminate all the tags that (he MH does not support, 
and references to any file that the MH is not capable of 
ltaiitllitig. For example, we can eliminate lite Unites tag, 
cascade style sheets, etc. for a BD& such as the PalmP.ilot. 
For such severely resource constrained MHs, the set or tags 
thai it can handle may be so small, that it is advantageous 
to strip at all unwanted tags al the proxy, and encode, the 
remaining tags using a few bits. 

By choosing the specific options, the user can use any or 
all of these transcoding methods depending ou the limita- 
tions of the client or. network connection, and can change it 
when resources change. Ibis makes our proxy very, adapt- 
able to serve the varying needs of Che user. For example, a 
nser on a laptop may want to only limit the sifce of video and 
audio files when spe is connected via a clow telephone. mo- 
dem, and remove this restriction when connected through 
Che etliernet. A : user on a PDA : , on the other Hand, will 
want to filter out everything except text and small images. 

5. Experimental results 

Our proxy server Is a modified version of a HTTP server, 
written in Peri. We are using an Apache server to store 
the preferences and to act as the :W\feW server capable of 
content- negotiation. VVc stored multiple formats of some 
files and requested them with different preference settings. 

An example of transcoding due to content negotiation: 

We set the following preferences for one computer. {A : 
desktop) Maximum Image file site = 20K Maximum Video 
file size = 500K 

The accept Headers added to its request were: Accept: 
huage/x-lgif, image/gif;q=0.6, video/ mpeg 

We requested for the page http:// bochi. cecs. missouri. 
edu: 9021/ demo. html 

Figure 2 shows the response received. 

We set the following preferences for another computer (A 
laptop) Maximum Image file size = 6K Maximum Video file 
size = 25K 



The accept headers added to its request were: 
Accept: imagejfc-sgif, image/gif;q=0.6 t v»deo/x-rrapg, 
videoj?mpeg;q=0.6 

We requested for the same page http:// bochl, cecs. 
missouri. edu: 9021/ demo. Html 
' Figure 3 shows the response received. 

An example of transcoding of images received from tiie 
server: 

For the same two computers, (same preferences set as 
above) we requested the page 

hiCpj^www.rnissouri.edu/'csacm U&e image tile existed 
only in gif format on the server. The image in the doc- 
ument: is small enough for. the desktop and timer, passed 
through without any reduction as seen in Figure 4. But 
the image is large for the laptop (larger than 6K). There- 
fore, tlie proxy reduced tlie resolution of the image as seen 
in Figure 5, 

To see some video files and their representative frames, 
please visit 
http:/;/meruxecs.missouri.eda/"sansanee^mpeg 

We kept different settings on two computers and ac- 
cessed the same web page to see the difference. For the 
first computer we set a large value for the maximum size 
of image and video files allowed, and sent accept headers 
to allow large gif files and mpeg movie files. Therefore, wc 
received large image files and the entire movie file. For the 
second computer, we limited the size of image and video 
files, and sent accept headers requesting for small images 
and. representative frames of mpeg files. Hence, we received 
smaller versions of: the images and only the representative 
frames for the movie file, ©ur request did not specify, the 
extension of the file name, and the file was available in mul- 
tiple formats. If. the size of the received image files is larger 
than the maximum size specified, the proxy scales it down 
either by size or by color as set by the user. 

Clearly., the results are best understood by experienc- 
ing the proxy, based model. We have made the proxy 
available on the web. it may be accessed at the t : RL 
http : % /nirvan a. cccs. missouri.edu :8GG1. A major drawback 
of the present implementation is its overhead. Since this 
is a demons tation prototype, it has been mostly in PERI;, 
and is thus may slow to execute. 

; • .. v . - f . . - • 

in this paper, we have presented a proxy based system 
(MGWSBR) to support web browsing horn mobile plat- 
forms. It foDows the elient-proxy-server model which is 
the Basis of most mobile applications and uses tho proxy 
to provide active transcoding. Proxies are mostly used for 
forwarding data between the mobile dieut and the station- 
ary, server. CEHe idea of using transcoding at the proxy to 
support mobility, is not new per se. Many proxy based sys- 
tems {5j t [!4},|t3] have been developed to provide web access 
to mobile users. However, they typically transcode the im- 
age data received from the VVWW server before sending 
it to the mobile client, and are often not configurable. In 
Mowser, we extend the notion of transcoding to both the 
upstream and the downstream traffic. More specifically, the 
upstream request is munged into a H!ETEj!l.t request, to 
make use of the content negotiation feature, and Accept 
headers are appended, to request for the document In the 
format most appropriate for the QoS parameters set by the 
mobile user. On the downstream, in addition to transcoding 
of images, we also provide the options of removing tlie Active 
content and transcoding HTM t. The mobile user can ralect 
any or all of these option*. This is to support the changing 



requirements of a wide variety of mobile host* ranging from 
a powerful notebook .with 233MH& GftU, 2GB RAM, which 
may. require only image antt video data transcoding, Co a 
PDA with 64KB dynamic RAM, which requires all possible 
transcoding and filtering of. data. 

!u ongoing wort, we ate extending Uie proxy to effec- 
tively use all (He preferences set by Che user to Emit or 
transform the data before serving the MH. Further, our 
proxy adds & performance overhead due to two reasons. 
First, it is written in Perl and uses netphm for the process- 
ing of image files. 3?he speed could be increased by writing 
optimized € code and image conversion routines. Second, 
messages go all the way up to the application layer in the 
proxy even if data jjust needs to be written from one socket 
to another. Research is going on in the IBM Watson liabs 
to avoid this by using 3!€P Splice [15] which' allows data to 
flow through without going to the application layer in the 
proxy if necessary and such techniques could eventually be 
integrated into our system. The overhead is justifiable be- 
cause we are trading proxy side 6PU cycles for the more 
expensive client side CPU cycles and network bandwidth. 
In experimental situations, it has been observed that extra 
time taken By the proxy is still less than the time needed 
to send untcansformed data on the wireless network. 
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Figure 1. Active Transcoding of HTTP Stream 
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Figure 3. Example of Content Negotiation: 
Response on a Resource poor Client 



Figure 5. Example of Image Transcoding: Re- 
sponse on a Resource poor Client 
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ABSTRACT 

There is a growing diversity of client devices that have 
access to the Internet. However, much of the content on 
the Internet cannot be handled by the devices that have 
limited communication, processing, storage and display ca- 
pabilities. In order to improve the utility of a wide range 
of client devices, we propose a network-based solution for 
tianscoding Internet content. The system uses an InfoPy- 
ramid for representing and transcoding video, images, au- 
dio and text. The InfoPyramid manipulates the content 
along the dimensions of fidelity and modality, and aggre- 
gates the methods for content analysis, translation, filter- 
ing and selection. The Info Pyramid utilises a policy engine, 
which incorporates user and publisher preferences, various 
transcoding policies, device descriptions, and real-time net- 
work constraints in order to adapt the Internet content to 
the client devices. 



1.1. Related work 

Recently, several systems have been developed for adapting 
Internet content to client devices. Fox, et at, developed 
a system for distilling, or compressing images that pass 
through an Internet proxy (3]. Other commercial systems 
such as Intel's Quick Web (4} and Spyglass 1 Prism [5] sim- 
ilarly compress the images that pass through the Internet 
service provider to speed-up download time. 

We have developed a content-based system for transcod- 
ing images [6]. This system retrieves and analyses images in 
the Internet and classifies them into image type and purpose 
classes in order to select appropriate methods for transcod- 
ing them. Since the system is limited to transcoding images, 
we propose a more powerful solution for Internet content 
that includes video, images, text and audio. The system 
transcodes the content along the dimensions of fidelity and 
modality in order to better adapt it to a large variety of 
cKent devices. 



1. INTRODUCTION 

Many new devices, such as personal digital assistants (PDAs), 
hand-held computers, Internet- ready phones, and WebTVs, 
are gaining access to the Internet. The capabilities of these 
devices to receive, process, store and display Internet con- 
tent varies widely. Given the variety of client devices, it is '=-* 
difficult for Internet content publishers to tailor contend tio^ 1 
the individual devices. <*- - — 

Internet content publishers do not have many options 
for customising the content. In some cases, the publish- 
ers manually generate secondary, text-only versions of Web 
pages that the users can select instead of the originals. 
Other mechanisms within the HTTP protocol allow the 
client to specify some dient attributes! such as the preferred 
language of the user, or the image, video, and audio formats 
supported by the client device. Using this information, the 
content serveT can automatically select and deliver content 
that is compatible with the client device and the user. 

In the case of cbent devices with minimal capabilities, 
such as pagers and cell phones, special markup languages 
are being developed, such as HDML ((1]). However, mech- 
anisms ttill need to be developed to automatically convert 
Internet content into these formats. The emerging XML 
markup language may improve the capabilities for adapt- 
ing content since separate- style sheets can be developed 
for client devices that determine how the content is dis- 
played [2). 



1.2. Outline 

In this paper, we propose a system for transcoding Internet 
content. In Section 2, we present an overview of the Internet 
content transcoding system. In Section 3, we present the 

. InfoPyramid framework for representing and manipulating 
z .video ,t images, ^audio- and . text. In Section 4, we describe 

^Trproicc^cs foriselectmg ith^ to adapt, 

thelcontent-to^the clients according to capabilities and pretT 

; erences. Finally, in Section 5, we describe the. deployment 
of the transcoding system in the Internet as transcoding 
proxies. 

2. INTERNET CONTENT TRANSCODER 

Figure I illustrates the proposed Internet content transcod- 
ing system. The system retrieves and analyses the Internet 
content and ingests it into the InfoPyramid format. A pol- 
icy engine gathers the capabilities of the client, the network 
conditions and the transcoding preferences of the user and 
publisher. This information is used to define the transcod- 
ing options for the client. The system then selects the out- 
put versions of the content and uses a library of content 
analysis, filtering, translation and manipulation routines to 
generate the content to be delivered to the client device. 

The Internet content transcoding system may be de- 
ployed at the server, proxy or client. Deployed at a proxy, 
the system is able to retrieve the Internet content, analyse 
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client prefs. 
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Content Internet Content Transcoder 

Figure 1: The Internet content transcoding system adapts the content to the capabilities of the client devices. 



and transcode it, and deliver the results to the client, on- 
the- fly. Deployed at the serve r, the system can be used 
in the content publication process. The system can pre- 
materialise the alternate versions of the Internet content 
and store them at the server. In this case, the system merely 
selects the versions of the content to deliver to the client. 
In some cases, the transcoding system can be deployed at 
the client to customise the content display, such as accord- 
ing the user preferences, as long as the client has sufficient 
capabilities. 

3. INFOPYRAM1D FRAMEWORK 

The InfbPyramid provides a general framework for han- 
dling the Internet content. It allows specialised methods to 
be plugged-iu for analysing, filtering, translating and ma- 
nipulating the Internet content. As depicted in Figure 2, 
the InfoPyiamid develops a conceptually redundant repre- 
sentation of the Internet content that aggregates multiple 
versions of the content along the dimensions of modality 
(video, image, text, and audio) and fidelity (which includes 
summarised and compressed versions} [7]. The translation 
and summarisation methods generate the alternate versions . 
of the content as needed. 

3.1. Translation and summarisation 

The translation methods convert the content between modal- 
ities, such as, text to audio, or video to images. On the 
other hand, the summarisation methods generate versions 
within the same modality, but with different fidelity. For 
example, the summarisation methods compress the images, 
summarise text, and extract and re-animate the key-frames 
from video. The translation and summarisation methods 
can be cascaded to change both the modality and fidelity 
of the content. In this case, a video can be converted to a 
small, reduced-color image. 

For each of the modalities, we describe some of the sum- 
marisation and translation methods that could be used to 
change the fidelity and modality of the content, respectively. 

1. Images: 

s Fidelity - Spatial sise reduction, color depth re- 
duction, lossy compression (6]. 




MODALITY 

Figure 2: The InfoPyramid aggregates multiple represen- 
tations of the content along the dimensions of fidelity and 
modality and unifies the methods for manipulating the con- 
tent. * 



• Modality - Images to text: related text, embed- 

±2 - • - d^-^S 4 ' semantic labels [8]. . . . , • . . ' 

— ' Fidelity-- Spatial sise, temporal sise, .playback- " 
rate, bit-rate. 

• Modality - Video to images: key-frames; Video 
to audio: sound track; Video to text: dosed 
captions, embedded text. 

y text: 

• Fidelity * Key-term extraction, text summa- 
risation, document headings extraction. 

• Modality - Text to audio: speech synthesis, 
Text to text: language translation (i.e., English 
to French). 

4. Audio: 

• Fidelity - Bit- rate seduction, stereo to mono. 

• Modality - Audio to text: speech recognition. 

In order to evaluate content alternatives, the system 
could assign content value scores to each of the content 
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alternatives. Using the content value •core*, the system is 
able to optimise the selection of the content according to 
the device capabilities, preference* end network conditions. 



4. TRANSCODING SYSTEM 

We propose that the transcoding system utilises s policy 
engine to evaluate the alternatives for adapting the content 
to the client devices. 



4.1. Policy Engine 

The policy engine would gather the capabilities of the client 
and the transcoding preferences of the user and publisher, 
and sense the network conditions to define the transcoding 
options for the client. In order adapt the Internet content to 
these devices, the transcoding proxy generates and selects 
version* of the content according to the policies, network 
and device constraints, and preferences. 



4.1.1. Content value scores 

The InfoPyramid system provides the mechanism for as- 
signing content value scores to the alternate versions of the 
content. In some cases, the content value scores are derived 
automatically by measuring the loss in fidelity that results 
from translating or summarising the content. For example, 
the content value scores can be linked to the distortion in- 
troduced from compressing the images or audio. Otherwise, 
the content value scores can be tied directly to the methods 
that manipulate the content. 




MODALITY 

Figure 3: Example Internet content value scores assigned 
for video. 

Figure $ illustrates examples of content value scores that 
can be assigned for transcoding video. In this example, 
the original video has the highest content value. Each ma- 
nipulation of the video along the dimension of fidelity or 
modality alters the content value. For example, converting 
the video to a sequence of images results in a small reduc- 
tion in content value. Converting the video to a highly 
compressed audio track produces a higher reduction in the 
content value. 



4.2.3. User and publisher preferences 

The content value scores comprise only part of the infor- 
mation that can be used in the content selection process. 
Both the publisher and user may have preferences for how 
the content is transcoded. Figure 4 illustrates some exam- 
ple preferences tor transcoding images. In this example, the 
preferred versions for the image content are the low-fidelity 
versions of the image, and the translations to text. 




MOOAUTY 

Figure 4: Internet content format preferences may be es- 
tablished by the publisher or user. This example illustrates 
the preferred content alternatives for images. Device con- 
straints may further score or eliminate some content alter- 
natives. 



4.1.3. Device constraints 

Various constraints of the devices affect the selection of the 
content. For example, many devices cannot handle video. 
In this case, the corresponding content alternatives can be 
eliminated (see Figure 4). Overall, the display, storage and 
processing capabilities of the client devices eliminate the 
selection of individual versions of the content. These also 
.place constraint*, on -the set of selections for aj¥eb docu- ' 
7nien£ toi e&g^g hw^ ^aYt^ge of only 
. 16 KBrthen.ihJsVplac« a hard limit WthYt^sisc of the™*' ' * 

versions of the content selected. ' *.-■■• 

4.1.4. Network constraints 

Similarly, the network would place constraints on the con- 
tent selection process. In general, the network constraints 
introduce a trade-off between content data site and load 
time. For example, if the user specifies a maximum load 
time for a page, then to accommodate this load time, the 
transcoder system must sense the end-to-end bandwidth to 
derive the target data sise for the content Then, the sys- 
tem can select the content of maximum content value that 
is within the target data sise. 

4.2. Content Selection 

We investigate more closely how the system could optimise 
the selection of the content alternatives. Consider the Web 
document with two objects, A and B, depicted in Figure 5. 
Let Ay = T ti (A) and B w = Tu(B) transcode A and B, 
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Figure 5: Example of content selection for a Web document 
consisting of two objects, A and B. 

respectively, to target modalities t and *, and fidelities j 
and l. Let V{Ai$) and V(Bht) be the content Taioe scores 
for Aij and Bu, and let S(Aij) and S(Bki) be the data sues 
of Aij and Bkt. Finally, let St be the maximum data sise for 
the content, which may have been derived from the user's 
specified maximum load-time and the network condition*. 

4.2.1. Maximum content value 

The content selection process selects A*j and B£, with max- 
imum content value fox a target data sise as follows: 

V(^)+V(Bi) = W*fY(A<s) + V{B u )) t and 
SM? y ) + 5(«,) < St. (1) 

4.2.2. Minimum load time 

Given a minimum acceptable content value Vr, the content 
alternatives A* s and Bl t of minimum data sise are given by: 

S{Al i )^S{Bl l ) m tqm($(A ij )+S(B u )) t and 

+ > V T . (2) 

4.2.3. Device constraints and preferences 

By extending this optimisation process, the content selec- 
tion system could incorporate the user (U[A^J) and pub- 
lisher (P{A}j)) preferences, and client device constraints 
(D(Aij)), to best adapt the content. In this case, the total 
score of each item k in the Web document is given by 

Z(A^) - w.vg + UuU* + u p P% + wtDfy, 

where w,, w„, u> p , and u d axe weighting factors assigned by 
the user or system. 

5. REAL-TIME, NETWORK-BASED TRANS COPING 

The transcoding system can be implemented in the net- 
work in the form of a transcoding proxy (TP) to perform 
transcoding cn-the-fly, sec Figure 6. The transcoding proxy 
system is designed to have a high bandwidth connection to 
the content provider. In most cases, the proxy has a low 
bandwidth connection to the client. As a result, reduc- 
ing the amount data through compression and transcoding 
at the proxy results in faster delivery, even when account- 
ing for the added time for content analysis, selection and 
transcoding [6]. 




Figure 6: Network-based transcoding of Internet content 
using transcoding proxies (TP). 



6. SUMMARY 

We proposed a system for the network-based transcoding 
of Internet content in order to improve the accessibility of a 
wide range of client devices to the content on the Internet. 
The transcoding system retrieves, analyses, and ingests the 
Internet content into an InfoPyramid representation. The 
Info Pj i amid provides a conceptual framework lor manip- 
ulating the content along the dimensions of modality and 
fidelity. The transcoding system selects the content from 
the InfoPyramid by assessing the various content alterna- 
tives to adapt the Internet content to the client devices. In 
this way, a wide range of client devices gain access to the 
large amounts of content on the Internet that is otherwise 
beyond their capabilities. 
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